United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OfTice 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


BRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/771,416 


02/05/2004 


Qi He Hong 


01263.000276. 


3175 



5514 7590 05/18/2007 

HTZPATRICK CELLA harper & SCINTO 
30 rockefeller PLAZA 
NEW YORK. NY 10112 



EXAMINER 



FUJITA, KATRINA R 



ART UNIT 



PAPER NUMBER 



2609 



MAIL DATE 



DELIVERY MODE 



05/18/2007 PAPER 

Please find below and/or attached an OfiHce communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



Office Action Summary 


Application No. 

10/771.416 


Appllcant(s) 

HONGETAL 


Examiner 

Katrine Fujita 


Art Unit 

26(D9 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER, FROIVI THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temi adjustment See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)n This action is FINAL. 2b)S This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-24 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1-24 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) K The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 05 February 2004 is/are: a)D accepted or b)E objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
aM All b)n Some * 0)0 None of: 

1 .13 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. , 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17,2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachments) 

1) S Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draflsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) EI Infonnation Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Infomial Patent Application 

Paper No(s)/Mail Date See Continuation Sheet 6) Q Other , 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20070426 



Continuation Sheet (PTOL-326) Appiication No. 10/771,416 



Continuation of Attachment(s) 3). Information Disclosure Statement(s) {PTO/SB/08), Paper No(s)/Mail Date :02/05/2004, 
12/20/2004. 02/04/2005. 



2 



Application/Control Number: 10/771.416 



Page 2 



Art Unit: 2609 

DETAILED ACTION 
Priority 

1 . Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 



Drawings 

2. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(4) 
because reference character "160" has been used to designate both the "segmentation 
parameter calculator*' and the "parameter setter". When read in light of the 
specification, it appears that the reference numeral for the "segmentation parameter 
calculator" should be "60" rather than "160". 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. Each drawing sheet 
submitted after the filing date of an application must be labeled in the top margin as 
either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the 
changes are not accepted by the examiner, the applicant will be notified and informed of 
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any required corrective action in the next Office action. The objection to the drawings 
will not be held in abeyance. 



Specification 

3. The abstract of the disclosure is objected to because it contains reference 
numerals from the drawings and exceeds 150 words in length. Correction is required. 
See MPEP§ 608.01(b). 

4. The disclosure is objected to because it contains an embedded hyperlink and/or 
other form of browser-executable code on page 3, line 12 and on page 37, line 24. 
Applicant is required to delete the embedded hyperlink and/or other form of browser- 
executable code. See MPEP § 608.01 . 

5. The disclosure is objected to because of the following informalities: 

On page 20, line 12, "surface modeler 70" should be -surface modeler 70 80-. 
On page 20, line 13, "texture generator 100" should be -texture generator 400 

90-. 

On page 34, line 20, "steps S9-34 to S9-34" should be -steps SO-34 S9-22 to 
S9-34-. 

Appropriate correction is required. 
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Claim Objections 

6. The following is a quotation of 37 CFR 1 .75(a): 

The specification must conclude with a claim particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention or discovery. 

7. Clainns 1-3, 10, 14, 15 and 20-22 are objected to under 37 CFR 1, 75(a), as failing 
to particularly point out and distinctly claim the subject matter which application regards 
as his invention or discovery. 

Claim 1 lacks an antecedent basis for "the two-dimensional projection" recited in 
line 9. The following will be assumed for examination purposes: - the a two- 
dimensional projection-. 

Claim 2 recites 'from an image" in line 2. It is unclear whether this is intended to 
be the same as or different from the "at least one image" recited in line 1 1 of claim 1 . 
The following will be assumed for examination purposes: -from a« the at least one 
image-. 

Claim 2 recites "selecting pixels" in line 4. It is unclear whether this is intended to 
be the same as or different from the "pixels" recited in line 1 , The following will be 
assumed for examination purposes: -selecting the pixels-. 

Claim 3 recites "pixels" in line 1 . It is unclear whether this intended to be the 
same as or different from the "pixels" recited in line 1 of claim 2. The following will be 
assumed for examination purposes: -the pixels-. 

Claim 3 recites "from an image" in line 2. It is unclear whether this is intended to 
be the same as or different from the "at least one image" recited in line 1 1 of claim 1 . 
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The following will be assumed for examination purposes: -from aft the at least one 
image--. 

Claim 3 recites "selecting pixels" in line 2. It is unclear whether this is intended to 
be the same as or different from the "pixels" recited in line 1 1 of claim 1 . The following 
will be assumed for examination purposes: -selecting the pixels-. 

Claim 10 lacks an antecedent basis for "the two-dimensional projection" recited in 
line 9. The following will be assumed for examination purposes: - the a two- 
dimensional projection-. 

Claim 14 recites "from an image" in line 2. It Is unclear whether this is intended 
to be the same as or different from the "at least one image" recited in line 12 of claim 
13. The following will be assumed for examination purposes: -from aft the at least one 
image-. 

Claim 14 recites "select pixels" in line 2. It is unclear whether this is intended to 
be the same as or different from the "pixels" recited in line 12 in claim 13. The following 
will be assumed for examination purposes: -select the pixels-. 

Claim 14 recites "selecting pixels" in line 4. It is unclear whether this is intended 
to be the same as or different from the "pixels" recited in line 12 of claim 13. The 
following will be assumed for examination purposes: -selecting the pixels-. 

Claim 15 recites "select pixels" in line 2. It is unclear whether this is intended to 
be the same as or different from the "pixels" recited in line 12 in claim 13. The following 
will be assumed for examination purposes: -select the pixels-. 
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Claim 15 recites "from an image" in line 2. It is unclear whether this is intended 
to be the same as or different from the "at least one image" recited in line 12 of claim 
13. The following will be assumed for examination purposes: -from an the at least one 
image-. 

Claim 15 recites "selecting pixels" in line 3. It is unclear whether this is intended 
to be the same as or different from the "pixels" recited in line 12 of claim 13. The 
following will be assumed for examination purposes: -selecting the pixels-. 

Claim 20 lacks an antecedent basis for "the two-dimensional projection" recited in 
line 9. The following will be assumed for examination purposes: - the a two- 
dimensional projection-. 

Claim 21 lacks an antecedent basis for "the two-dimensional projection" recited in 
line 9. The following will be assumed for examination purposes: - the a two- 
dimensional projection-. 

Claim 22 lacks an antecedent basis for "the two-dimensional projection" recited in 
line 9. The following will be assumed for examination purposes: - the a two- 
dimensional projection-. 



Claim Rejections - 35 USC § 101 



8. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, nnachine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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The USPTO "Interim Guidelines for Examination of Patent Applications for 
Patent Subject Matter Eligibility" (Official Gazette notice of 22 November 
2005), Annex IV, reads as follows: 

Claims that recite nothing but the physical characteristics of a form of energy, such as a frequency, 
voltage, or the strength of a magnetic field, define energy or magnetism, per se, and as such are 
nonstatutory natural phenomena. O'Reilly. 56 U.S. (15 How.) at 112-14. Moreover, it does not appear 
that a claim reciting a signal encoded with functional descriptive material falls within any of the 
categories of patentable subject matter set forth in Sec. 101 . 

... a signal does not fall within one of the four statutory classes of Sec. 101 . 

... signal claims are ineligible for patent protection because they do not fall within any of the four 
statutory classes of Sec. 101. 



9. Claim 24 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 24 defines a signal with 
descriptive material. While "functional descriptive material" may be claimed as a 
statutory product (i.e., a "manufacture") when embodied on a tangible computer 
readable medium, a signal embodying that same functional descriptive material is 
neither a process (I.e., a series of steps per se.) nor a product (i.e., a tangible "thing") 
and therefore does not fall within one of the four statutory classes of § 101 . Rather, a 
"signal" is a form of energy, in the absence of any physical structure or tangible 
material. 



Claim Rejections - 35 USC § 102 



(a) the Invention was known or used by others in this country, or patented or described In a printed 
publication In this or a foreign country, before the invention thereof by the applicant for a patent. 



Application/Control Number: 1 0/771 .41 6 Page 8 

Art Unit: 2609 



(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent In the United 

States. 



(e) the invention was described In (1) an application for patent, published under section 122(b), by 
another filed In the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



10. Claims 1,2, 4-7. 10, 13, 14, 16-22. 23/1,2, 4-7,10 and 24/1, 2, 4-7. 10 are 
rejected under 35 U.S.C. 102(b) as being anticipated by Fujiwara (US 2002/0150288). 

Regarding claims 1, 10, 13, and 20, Fujiwara discloses a method and apparatus 
for processing data defining a plurality of images of an object recorded at different 
positions and orientations ("method and device for processing an image in which a 
portion corresponding to a subject is extracted from plural shot image" at paragraph 
0003, line 1 ) and data defining the positions and orientations to generate data defining a 
three-dimensional computer model of the object ("reconstructs the shape in accordance 
with the extracted object areas" at paragraph 0023, line 20; "acquires shoot condition 
data D1 that indicate the positional relationship between the digital camera 10 and the 
object 90" at paragraph 0023, line 21) comprising: 

a volume definer (figure 8, numeral 601) operable to define a volume in three- 
dimensional space ("the three-dimensional shape data are initialized" at paragraph 
0036, line 1) enclosing the object ("the area where the voxels are arranged is set 
sufficiently larger than the area of the virtual space in which the object 90 is expected to 
exist" at paragraph 0036, line 4); 
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a volume projector (figure 6, numeral 204) operable to determine a two- 
dimensional projection of the volume ("visual volume is projected" at paragraph 0047, 
line 1) in at least one of the images ("lines of sight passing through the boundary, are 
projected onto the image" at paragraph 0047, line 3); 

a pixel selector (figure 6, numeral 210) operable to select pixels ("pixel value 
corresponding to the binary image H| is "1" and the pixel value corresponding to the 
image Ij satisfies the following condition is determined" at paragraph 0055, line 2) from 
at least one image in dependence upon the volume projection therein ("line of sight 
projected onto image 1" at paragraph 0047, line 7); 

a segmentation parameter definer (figure 6, numeral 210) operable to determine 
segmentation parameters ("||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, line 8) in 
dependence upon at least one image property of the selected pixels ("Ck denotes the 
color vector of the pixel Pk" at paragraph 0056, line 4), the segmentation parameters 
comprising parameters for distinguishing between subject object image data and other 
image data during segmentation processing ("pixel having the binary image value of "1" 
is a pixel that was estimated to belong to the object area" at paragraph 0057, line 3); 

an image data segmenter (figure 6, numeral 210) operable to process the image 
data to segment image data relating to the object from other image data ("pixel having 
the binary image value of "1" is a pixel that was estimated to belong to the object area" 
at paragraph 0057, line 3) in at least some of the images ("image 1" at paragraph 0055, 
line 4) using the generated segmentation parameters ("satisfies the following condition 
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is determined" at paragraph 0055, line 4; "||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, 
line 8); and 

a three-dimensional computer model data generator (figure 3, numeral 120) 
operable to generate data defining a three-dimensional computer model of the object 
("the three-dimensional shape data are displayed" at paragraph 0042, line 1) using the 
results of the segmentation processing and the data defining the positions and 
orientations at which the images were recorded ("estimates the three-dimensional 
shape of the object 90 in accordance with the object areas extracted from plural, 
images" at paragraph 0023, line 29). 

Regarding claims 2 and 14, Fujiwara discloses a method and apparatus wherein 
the pixels are selected ("pixel value corresponding to the binary image Hj is "1" and the 
pixel value corresponding to the image Ij satisfies the following condition is determined" 
at paragraph 0055, line 2) from the at least one image by determining the position of the 
outer perimeter ("enclosed by the lines of sight passing the object area 710 in the 
processed image 71[j-th Q-l, 2,... ,1-1) Image IJ and the boundary thereof at paragraph 
0040, line 4) of the volume projection in the image ("line of sight projected onto image I " 
at paragraph 0047, line 7) and selecting the pixels in dependence upon the determined 
outer perimeter position ("area becomes an intersection area v of the projection image 
areas" at paragraph 0049, line 5; figure 4B, numeral 757). 

Regarding claims 4 and 16, Fujiwara discloses a method and apparatus wherein 
the apparatus is operable to repeat the processing operations at least once ("it is 
decided whether the subsequent process is repeated or not" at paragraph 0043, line 1), 
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and wherein, on the second and each subsequent time the operations are performed 
("process goes back to Step #1 1 0 for taking a picture of another position of posture 
different from the previous one" at paragraph 0043, line 8), the volume definer (figure 8, 
numeral 601) is arranged to define the volume to be the three-dimensional computer 
model of the object generated a previous time the operations were performed ("if the 
user decided that the shape is not sufficiently reproduced, he or she instructs the 
computer 60 to continue the process" at paragraph 0043, line 6; the subsequent 
processing is based on the constructed volume rather than an initialized volume as 
defined in figure 3, numeral 108). 

Regarding claims 5 and 17, Fujiwara discloses a method and apparatus wherein 
the image data segmenter (figure 6, numeral 210) is operable to perform segmentation 
processing using the generated segmentation parameters ("satisfies the following 
condition is detennined" at paragraph 0055, line 4; "||Ck- CbII>tb or ||CkCr||>tr" at paragraph 
0055, line 8) only on image data within the projection of the volume in the image ("pixel 
value corresponding to image 1" at paragraph 0055, line 3; "lines of sight passing 
through the boundary are projected onto the image li" at paragraph 0047, line 3), and to 
classify the image data outside the projection of the volume as image data which does 
not relate to the object ("pixel value corresponding to the determined pixel area of "1" 
and the other pixel value of "0" is generated" at paragraph 0055, line 5). 

Regarding claims 6 and 18, Fujiwara discloses a method and apparatus wherein 
the segmentation parameter definer (figure 6, numeral 210) is operable to determine the 
segmentation parameters ("||Ck- Cb||>tb or ||CkCr|I>tr" at paragraph 0055, line 8) in 



Application/Control Number: 1 G/771 ,416 Page 1 2 

Art Unit: 2609 

dependence upon the value of at least one colour component of each selected pixel 
("reference character Cr denotes the color vector of the pixel Pk" at paragraph 0056, line 
4). 

Regarding claims 7 and 19, Fujiwara discloses a method and apparatus 
wherein: 

each image to be processed (figure 2, numeral 70) shows the object together 
with a calibration object (figure 1, numeral 40; "camera calibration tool 40" at paragraph 
0029, line 4) and the data defining the positions and orientations of the images 
("acquires shoot condition data D1 that indicate the positional relationship between the 
digital camera 10 and the object 90" at paragraph 0023, line 21) defines the positions 
and orientations of the images and the position of the calibration object in the same 
three-dimensional coordinate system ("display area 621 displays an image 70 inputted 
from the digital camera" at paragraph 0025, line 3; "gives a geometric relationship 
between the world coordinates and the pixel coordinate of the image" at paragraph 
0029, line 8); and 

the volume definer (figure 3, numeral 108) is operable to define the volume 
enclosing the object ("the area where the voxels are arranged is set sufficiently larger 
than the area of the virtual space in which the object 90 is expected to exist" at 
paragraph 0036, line 4) in the three-dimensional coordinate system of the images and 
calibration object ("the three-dimensional shape data are initialized" at paragraph 0036, 
line 1) in dependence upon the calibration object ("cursor on one of the grid points of 
the camera calibration tool 40... after repeating this operation for plural grid points, the 
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user instructs the computer 60 to calculate a projection matrix Po" at paragraph 0029, 
line 3). 

Regarding claim 21, Fujiwara discloses an apparatus for processing data 
defining a plurality of images of an object recorded at different positions and orientations 
and data defining the positions and orientations to generate data defining a three- 
dimensional computer model of the object, the apparatus comprising: 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for constmcting a volume around the object) defining a volume in three- 
dimensional space ("the three-dimensional shape data are initialized" at paragraph 
0036, line 1) enclosing the object ("the area where the voxels are arranged is set 
sufficiently larger than the area of the virtual space in which the object 90 is expected to 
exist" at paragraph 0036, line 4); 

means for (figure 1, numeral 60, which is equivalent to applicant's disclosed 
apparatus for projecting the volume into an image) determining the two-dimensional 
projection of the volume ("visual volume is projected" at paragraph 0047, line 1 ) in at 
least one of the images ("lines of sight passing through the boundary are projected onto 
the image" at paragraph 0047, line 3); 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for choosing pixels from the image) selecting pixels ("Among the pixels pk 
(k=1 ,2,. . .Nj) of the image h, a pixel area that satisfies either of the two following 
conditions is determined" at paragraph 0049, line 2) from at least one image in 
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dependence upon the volume projection therein ("line of sight projected onto image Ij" at 
paragraph 0047, line 7); 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for creating segmentation parameters for segmentation) determining 
segmentation parameters ("||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, line 8) in 
dependence upon at least one image property of the selected pixels ("Ck denotes the 
color vector of the pixel Pk" at paragraph 0056, line 4), the segmentation parameters 
comprising parameters for distinguishing between subject object image data and other 
image data during segmentation processing ("pixel having the binary image value of "1" 
is a pixel that was estimated to belong to the object area" at paragraph 0057, line 3); 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for segmenting the image) processing the image data to segment image data 
relating to the object from other Image data ("pixel having the binary image value of "1'' 
is a pixel that was estimated to belong to the object area" at paragraph 0057, line 3) in 
at least some of the images ("image Ij" at paragraph 0055, line 4) using the generated 
segmentation parameters ("satisfies the following condition is determined" at paragraph 
0055, line 4; "||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, line 8); and 

means for figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for constructing the 3D image model) generating data defining a three- 
dimensional computer model of the object ("the three-dimensional shape data are 
displayed" at paragraph 0042, line 1) using the results of the segmentation processing 
and the data defining the positions and orientations at which the images were recorded 
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("estimates the three-dimensional shape of the object 90 in accordance with the object 
areas extracted from plural images" at paragraph 0023. line 29). 

Regarding claim 22, Fujiwara discloses an apparatus for processing data 
defining a plurality of images of an object recorded at different positions and orientations 
and data defining the positions and orientations to segment image data relating to the 
object from other image data in the images, the apparatus comprising: 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for constructing a volume around the object) defining a volume in three- 
dimensional space ("the three-dimensional shape data are initialized" at paragraph 
0036, line 1 ) enclosing the object ("the area where the voxels are arranged is set 
sufficiently larger than the area of the virtual space in which the object 90 is expected to 
exist" at paragraph 0036, line 4); 

means for (figure 1, numeral 60, which is equivalent to applicant's disclosed 
apparatus for projecting the volume into an image) determining the two-dimensional 
projection of the volume ("visual volume is projected" at paragraph 0047, line 1) in at 
least one of the images ("lines of sight passing through the boundary are projected onto 
the image" at paragraph 0047, line 3); 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for choosing pixels from the image) selecting pixels ("Among the pixels Pk 
(k=1,2,...Nj) of the image Ij, a pixel area that satisfies either of the two following 
conditions is determined" at paragraph 0049, line 2) from at least one image in 
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dependence upon the volume projection therein ("line of sight projected onto image Ij" at 
paragraph 0047, line 7); 

means for (figure 1 , numeral 60, which is equivalent to applicant's disclosed 
apparatus for creating segmentation parameters for segmentation) determining 
segmentation parameters ("||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, line 8) in 
dependence upon at least one Image property of the selected pixels ("Ck denotes the 
color vector of the pixel Pk" at paragraph 0056, line 4), the segmentation parameters 
comprising parameters for distinguishing between subject object image data and other 
image data during segmentation processing ("pixel having the binary image value of "1" 
is a pixel that was estimated to belong to the object area" at paragraph 0057, line 3); 
and 

means for (figure 1, numeral 60, which is equivalent to applicants disclosed 
apparatus for segmenting the image) processing the image data to segment image data 
relating to the object from other image data ("pixel having the binary image value of "1" 
is a pixel that was estimated to belong to the object area" at paragraph 0057, line 3) in 
at least some of the images ("image Ij" at paragraph 0055, line 4) using the generated 
segmentation parameters ("satisfies the following condition is determined" at paragraph 
0055, line 4; "||Ck- Cb||>tb or ||CkCr||>tr" at paragraph 0055, line 8). 

Regarding claims 23/1, 23/2, 23/4-7 and 23/10, Fujiwara discloses a storage 
medium storing computer program instructions (figure 1, numerals 67 and 68) to 
program a programmable processing apparatus to become operable to perform the 
method as described in claims 1,2, 4-7 and 10 ("A program recorded in a CD-ROM 67 
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or another medium 68 is installed into the computer 60. The program makes the 
computer 60 work as an image data processor and modeling device" at paragraph 
0023, line 12). 

Regarding claims 24/1, 24/2, 24/4-7 and 24/10, Fujiwara discloses a signal 
carrying computer program instructions (electrical signal that results from reading 
contents of figure 1, numerals 67 and 68) to program a programmable processing 
apparatus to become operable to perform the methods as described in claims 1 , 2, 4-7 
and 10 ("A program recorded in a CD-ROM 67 or another medium 68 is installed into 
the computer 60. The program makes the computer 60 work as an image data 
processor and modeling device" at paragraph 0023, line 12). 

Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not Identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 3, 1 5, 23/3 and 24/3 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Fujiwara and Kutulakos et al. ("A Theory of 
Shape...", Computer Science Technical Report). 
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Regarding claims 3 and 15, Fujiwara discloses a method and apparatus wherein 
the pixels are selected ("pixel value corresponding to the image \\ satisfies the following 
condition is detemiined" at paragraph 0055, line 3) from the at least one image by 
detemnining the position of the outer perimeter ("enclosed by the lines of sight passing 
the object area 710 in the processed image 710-th (j-1.2,...,i-1) image IJ and the 
boundary thereof at paragraph 0040, line 4) of the volume projection in the Image ("line 
of sight projected onto image l|" at paragraph 0047, line 7). 

Fujiwara does not disclose selecting the pixels from a band adjacent the outer 
perimeter of the volume projection. 

Kutulakos et al. discloses a method ("simple, discrete algorithm that "carves" 
space" at section 4, paragraph 1 , line 1 ) and apparatus ("compute V* on a SGI 02 
R10000/175MHZ workstation" at section 5, paragraph 4, line 1 1) for reconstructing a 3D 
object using multiple images ("3D scene from multiple photographs" at section 1, 
paragraph 1 , line 1 ) wherein pixels are selected ("selecting the voxel" at section 4, 
paragraph 4, line 1) from at least one image ("all photographs in ViSv(v)" at section 4, 
paragraph 4. line 12) by selecting the pixels from a band (the surface of V consists of a 
voxel wide band) adjacent the outer perimeter of the volume projection ("voxel v is 
found on the surface of V" at section 4, paragraph 4, line 1 1 ; the surface of V is adjacent 
the outer perimeter of the projection). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the pixel selection of Fujiwara using the voxel examination 
taught by Kutulakos et al. as described above, such that the "volume is guaranteed to 
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converge to the maximal photo-consistent shape" (Kutulakos et al. at section 4, 
paragraph 4, line 4). 

Regarding claim 23/3, Fujiwara discloses a storage medium storing computer 
program instructions (figure 1, numerals 67 and 68) to program a programmable 
processing apparatus to become operable to perform the method as described in claim 
3 ("A program recorded in a CD-ROM 67 or another medium 68 is installed into the 
computer 60. The program makes the computer 60 work as an image data processor 
and modeling device" at paragraph 0023, line 12). 

Regarding claim 24/3, Fujiwara discloses a signal carrying computer program 
instructions (electrical signal that results from reading contents of figure 1, numerals 67 
and 68) to program a programmable processing apparatus to become operable to 
perform the methods as described in claim 3 ("A program recorded in a CD-ROM 67 or 
another medium 68 is installed into the computer 60. The program makes the computer 
60 work as an image data processor and modeling device" at paragraph 0023, line 12). 

1 3. Claims 8,9,11, and 1 2 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over the combination of Fujiwara and Chang et al. (US 7,046,840). 

Regarding claim 8, Fujiwara discloses the elements of claim 1 as described in 
the 102 rejection above. 

Fujiwara does not disclose a method further comprising generating a signal 
carrying data defining the generated three-dimensional computer model. 
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Chang et al. discloses a method of processing data ("method for constructing a 
three-dimensional model" at col. 2, line 65) defining a plurality of images of an object 
recorded at different positions and orientations ("Photographs from several perspectives 
are taken of the object" at col. 3, line 13) and data defining the positions and 
orientations ("locations of the features in the images" at col. 3, line 4) to generate data 
defining a three-dimensional computer model of the object ("detemiine a three- 
dimensional model of the object" at col. 3, line 7), the method comprising generating a 
signal ("exported for use in other software" at col. 13, line 48; when exporting the data, 
an electrical signal is produced within the computer) carrying data defining the 
generated three-dimensional computer model ("reconstructed 3D model" at col. 13, line 
48). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the computer body of Fujiwara using the signal generator taught 
by Kochi et al. as described above, to increase versatility of the model data by enabling 
it to be utilized by other software programs. 

Regarding claim 11, Fujiwara discloses the elements of claim 10 as described in 
the 102 rejection above. 

Fujiwara does not disclose a method further comprising generating a signal 
carrying data defining the silhouette of the subject object in each of the at least some 
images. 

Chang et al. discloses a method of processing data ("method for constructing a 
three-dimensional model" at col. 2, line 65) defining a plurality of images of an object 
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recorded at different positions and orientations ("Photographs from several perspectives 
are taken of the object" at col. 3, line 13) and data defining the positions and 
orientations ("locations of the features in the images" at col. 3, line 4) to generate data 
defining a three-dimensional computer model of the object ("determine a three- 
dimensional model of the object" at col. 3, line 7), the method comprising generating a 
signal ("exported for use in other software" at col. 13, line 48; when exporting the data, 
an electrical signal is produced within the computer) carrying data defining the 
silhouette of the subject object in each of the at least some images ("reconstructed 3D 
model" at col. 13, line 48; the 3D model shows the outline, or silhouette, of the object). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the computer body of Fujiwara using the signal generator taught 
by Kochi et al. as described above, to increase versatility of the model data by enabling 
it to be utilized by other software programs. 

Regarding claims 9 and 12, Fujiwara discloses the elements of claims 1 and 10 
as described in the 102 rejections aboVe. 

Fujiwara does not disclose a method further comprising making a recording of 
the signal either directly or indirectly. 

Chang et al. discloses a method of processing data ("method for constructing a 
three-dimensional model" at col. 2, line 65) defining a plurality of images of an object 
recorded at different positions and orientations ("Photographs from several perspectives 
are taken of the object" at col. 3, line 13) and data defining the positions and 
orientations ("locations of the features in the images" at col. 3, line 4) to generate data 
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defining a three-dimensional computer model of the object ("determine a three- 
dimensional model of the object" at col. 3, line 7), the method comprising making a 
recording ("format conversion process can modify the complexity of the 3D model... to 
meet the quality or rendering speed requirements of a particular user or software 
package" at col. 13. line 52; the data is copied over after export) of the signal ("exported 
for use in other software" at col. 13, line 48; when exporting the data, an electrical signal 
is produced within the computer) either directly or indirectly. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the computer body of Fujiwara using the signal recording taught 
by Kochi et al. as described above, to increase versatility of the model data by enabling 
it to be utilized by other software programs. 



Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US 6,563,499 and US 6,455,835 are each pertinent as teaching 
computer based 3D reconstruction systems and methods. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Katrina Fujita whose telephone number is (571) 270- 
1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8-4:30pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian P. Werner can be reached on (571 ) 272-7401 . The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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